Electrical charging system, electrical charging controlling method, robot apparatus, electrical charging device, electrical charging controlling program and recording medium

ABSTRACT

A robot apparatus  1  is to be electrically charged autonomously. An electrical charging device  100  is provided with two markers, namely a main marker  118  and a sub-marker  119 , and the heights of the markers are pre-stored in the robot apparatus. When the robot apparatus  1  is to find the direction and the distance to the electrical charging device  100 , a CCD camera  20  finds the direction vector of the marker from the photographed image. This direction vector is transformed into a position vector of a camera coordinate system {c} and further into a position vector of the robot coordinate system {b}. The coordinate in the height-wise direction in the robot coordinate system {b} is compared to the pre-stored height to find the distance between the markers and the robot apparatus and the direction of the robot apparatus.

This is a continuation of application Ser. No. 10/400,094, filed Mar.26, 2003, now U.S. Pat. No. 6,914,403 the entirety of which isincorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention relates to an electrical charging system, an electricalcharging controlling method, a robot apparatus, an electrical chargingdevice, an electrical charging controlling program, and a recordingmedium. This application claims priority of Japanese Patent ApplicationNo.2002-090026, filed on Mar. 27, 2002, the entirety of which isincorporated by reference herein.

2. Description of Related Art

Recently, development in practically useful robots, supporting the humanlife as a partner, that is supporting the human activities in variousaspects of our everyday life, such as in living environment, isprogressing. In distinction from the industrial robots, thesepractically useful robots are endowed with the ability to learn forthemselves the method for adaptation to human beings with variablepersonalities, or to variable environments, in the variegated aspects ofour everyday life. For example, pet-type robots, simulating the bodilymechanism or movements of animals, such as quadruples, e.g., dogs orcats, or so-called humanoid robots, simulating the bodily mechanism ormovements of animals erected and walking on feet, such as human beings,are already being put to practical use.

As compared to the industrial robots, the above-described robotapparatus are able to perform variable entertainment-orientedoperations, and hence are sometimes called entertainment robots.

Meanwhile, since the above-described pet type robot or the humanoidrobot behaves autonomously, it is connected with for example a cable toa power supply, and hence the driving of the robot apparatus proveshindrance to the behavior of the robot apparatus. In this consideration,such robot apparatus includes an built-in battery, as a power supplysource.

However, since the quantity of the power that can be stored in thebattery is limited, it is necessary with the conventional robotapparatus to charge the battery periodically. If this periodic chargingis not made, the behavior of the robot apparatus no longer occurs due tobattery power down. Moreover, this periodic battery charging iscumbersome for the user.

SUMMARY OF THE INVENTION

It is therefore an object of the present invention to provide anelectrical charging system and an electrical charging controlling methodfor a robot apparatus, in which electrical charging may take placeautonomously, a robot apparatus, an electrical charging device, anelectrical charging controlling program, and a recording medium.

In one aspect, the present invention provides an electrical chargingsystem including a robot apparatus and an electrical charging device,the robot apparatus having built-in a power supply unit for driving andadapted for acting autonomously responsive to an inner state, with theelectrical charging device electrically charging the power supply unit,wherein the electrical charging device includes a marker provided at apredetermined position, the robot apparatus including imaging means forimaging a surrounding state, storage means having the information on theposition of the marker pre-stored therein, position vector calculatingmeans for calculating the distance and the direction to the marker fromthe information on the marker position stored in the storage means andfrom an image including the marker position, photographed by the imagingmeans, and operation controlling means for performing control forcausing movement of the robot apparatus towards the electrical chargingdevice depending on the distance and the direction to the markercalculated by the position vector calculating means.

In another aspect, the present invention provides a robot apparatusincluding imaging means for imaging a surrounding state, storage meanshaving the information on the position of a marker provided to anelectrical charging device electrically charging the power supply unit,position vector calculating means for calculating the distance and thedirection to the marker from the information on the marker positionstored in the storage means and from an image including the markerposition, photographed by the imaging means, and operation controllingmeans for performing control for causing movement of the robot apparatustowards the electrical charging device depending on the distance and thedirection to the marker calculated by the position vector calculatingmeans.

In still another aspect, the present invention provides an electricalcharging device for a robot apparatus having a charging batterycomprising an outer casing, the height of which is gradually decreasedfrom one longitudinal end to the opposite side longitudinal end thereof,connection means electrically connected to the robot apparatus forsupplying at least the electrical power, and a marker recognizable bythe robot apparatus.

Preferably, at least two of the markers are provided at differentpositions of the electrical charging device, and the storage meanshaving stored therein the position relationships of said markers and themounting positions thereof to the electrical charging device.

In still another aspect, the present invention provides an electricalcharging controlling method including an imaging step of imaging asurrounding state, storage means having the information on the positionof the marker pre-stored therein, a position vector calculating step ofcalculating the distance and the direction to the marker from an imageof the marker provided to en electrical charging device, photographed bythe imaging step, and from information on the position of the markerpre-stored in storage means, and a movement controlling step ofperforming control for causing movement of the robot apparatus towardsthe electrical charging device depending on the distance and thedirection to the marker calculated by the position vector calculatingstep.

The present invention also provides a control program for having therobot apparatus carry out the electrically charging controlling method,and a recording medium having recorded this control program.

The electrical charging system of the present invention includes anelectrical charging device including a marker provided at apredetermined position, with the robot apparatus including imaging meansfor imaging a surrounding state, storage means having the information onthe position of the marker pre-stored therein, position vectorcalculating means for calculating the distance and the direction to themarker from the information on the marker position stored in the storagemeans and from an image including the marker position, photographed bythe imaging means, and operation controlling means for performingcontrol for causing movement of the robot apparatus towards theelectrical charging device depending on the distance and the directionto the marker calculated by the position vector calculating means.

Thus, with the electrical charging system according to the presentinvention, it is possible for the robot apparatus to locate theelectrical charging device and to be moved towards the electricalcharging device to carry out electrical charging autonomously.

With the robot apparatus according to the present invention, theposition vector calculating means calculates the distance and thedirection to the marker, based on the marker position information storedin the storage means, and the image including the marker position,photographed by the imaging means, and accordingly locates theelectrical charging device to proceed towards it to carry out autonomouscharging.

With the robot apparatus according to the present invention, the robotapparatus has the visually recognizable marker, whereby the position ofthe electrical charging device can be recognized by the robot apparatus.

Moreover, with the electrical charging controlling method according tothe present invention, the distance and the direction to the markerprovided to the electrical charging device, are calculated from theimage of the marker photographed in the imaging step, and from theinformation on the marker position pre-stored in the storage means, inthe position vector calculating step, and the robot apparatus iscontrolled to be moved towards the electrical charging device, dependingon the distance and the direction calculated, in the operationcontrolling step, the robot apparatus locating the electrical chargingdevice and proceeding towards the electrical charging device to effectelectrical charging autonomously.

By recording the control program for having a robot apparatus executethe electrical charging controlling method on a recording medium, and byproviding this recording medium, it is possible for the robot apparatusto locate the electrical charging device to proceed towards theelectrical charging device to effect electrical charging autonomously.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a perspective view showing an electrical charger embodying thepresent invention.

FIG. 2 is a perspective view showing another electrical chargerembodying the present invention.

FIG. 3 is a perspective view showing still another electrical chargerembodying the present invention.

FIG. 4 is a schematic view showing yet another electrical chargerembodying the present invention.

FIG. 5 is a perspective view showing a robot apparatus embodying thepresent invention.

FIG. 6 shows an inner structure of the robot apparatus embodying thepresent invention.

FIG. 7 is a schematic view showing a coordinate system used in automaticelectrical charging.

FIG. 8 shows the relative distance and the relative angle of a robotcoordinate system and a station coordinate system.

FIG. 9 shows an entrance point and a passage point.

FIG. 10 schematically shows the operation of a robot apparatus inautomatic electrical charging.

FIG. 11 is a flowchart showing automatic electrical charging.

FIG. 12 illustrates the relationship between the norm of the positionvector and the number of frames.

FIG. 13 shows the software structure of a control program in the robotapparatus.

FIG. 14 shows the structure of a middleware layer in the softwareenclosed in the robot apparatus.

FIG. 15 shows the structure of an application layer in the softwareenclosed in the robot apparatus.

FIG. 16 shows the structure of a behavior model library in the softwareenclosed in the robot apparatus.

FIG. 17 is a status transition diagram for determining the behavior ofthe robot apparatus.

FIG. 18 shows status transition conditions for determining the behaviorof the robot apparatus.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

The electrical charging system according to the present invention ismade up by a robot apparatus, having a built-in power supply and whichis able to act autonomously responsive to its inner-state, and anelectrical charger for electrically charging a power supply. In thiselectrical charging system, the robot apparatus performs autonomousmovements towards the electrical charger to start the electricalcharging automatically. The robot apparatus is able to recognize theposition of the electrical charger because the electrical charger isprovided with a marker. The robot apparatus memorizes the position ofthis marker and finds the direction and the distance to the electricalcharger from the photographed image of the marker and the pre-storedmarker position.

In the present embodiment, the structure of the electrical charger 100is explained first and subsequently the robot apparatus is explained indetail. Referring to FIGS. 1 to 3, an illustrative structure of theelectrical charger 100 according to the present invention is explainedin detail.

The electrical charger 100 includes a robot apparatus support 111, onwhich to load the robot apparatus 1, a display unit 112 fordemonstrating e.g., the electrical charging state of the robotapparatus, a front cover 113 for partially covering the display unit112, an operating unit 114, and a main marker 118 and a sub-marker 119,as indices for indicating the presence of the electrical charger.

The electrical charger 100 is formed as a semi-droplet shaped device,the height of which is lowered gradually along a curved surface from onelongitudinal end to the other longitudinal end of the electricalcharging device. The display unit 112 is provided with an end face of ahigher height. The surface of the electrical charger 100 provided withthe display unit 112 is termed a front surface, while the lowered endthereof is termed a rear end.

The highest point of the electrical charger 100 is of approximately thesame height as that up to the belly of the robot apparatus 1 upstandingstill on its four legs. The transverse length of the electrical charger100 is selected to be less than the length between left and right legsof the robot apparatus 1.

In the electrical charger 100, the robot apparatus support 111 is shapedas a bowl lying along the belly of the robot apparatus 1. The electricalcharger 100 is provided with a connection unit 115 and a loadingdetection unit 116 on the major surface of the robot apparatus support111.

The connection unit 115 is provided to a position corresponding to amating connection unit provided on the robot apparatus 1 when the robotapparatus 1 is loaded in position. The connection unit 115 has thefunction as a connector for power supply and information exchange, in amanner not shown. When the robot apparatus 1 is set on the electricalcharger, the connection unit 115 is contacted with the mating connectionunit provided on the robot apparatus 1 to supply the power to the robotapparatus 1. The connection unit 115 also receives the information onthe electrically charging by the charging battery loaded on the robotapparatus 1 or the information on the inner state thereof.

The loading detection unit 116 is mounted at a position contacted by therobot apparatus 1 when the robot apparatus 1 is mounted on theelectrical charger. The loading detection unit 116 is protruded in aperpendicular direction to the major surface of the robot apparatussupport 111 and is elastically biased in the protruding direction, sothat, when the robot apparatus 1 is mounted on the robot apparatussupport 111, the loading detection unit is pushed into the inside of theelectrical charger 100 by the belly portion of the robot apparatus 1.Thus, a loading detection device 119 detects that the robot apparatus 1has been placed in position. On detecting that the robot apparatus 1 hasbeen loaded in position, the loading detection device 119 commenceselectrical charging to acquire the information on the inner state of therobot apparatus 1.

The display unit 112 is e.g., a liquid crystal display and demonstratesat least the information on the electrical charging of the chargingbattery mounted on the robot apparatus 1, the information on the innerstate of the robot apparatus 1 acquired through an information exchangeconnector, and other various information. The display unit 112 alsodemonstrates the information on the electrical charging of a chargingbattery as a sub-battery charged in a charging unit 117 as laterexplained.

The front cover 113 includes an opening in a portion corresponding tothe display unit 112 on the front surface. The front cover 113 is formedas a subsequently L-shaped section for overlying the operating unit 114and display means provided on an upper surface.

The front cover 113 is mounted to a preset location near the bottomsurface of the electrical charger 100 and may be opened/closed on a lineof extension of the longitudinal direction of the main body unit of theelectrical charger 100.

The operating unit 114, used, for example, for selecting the function ofthe electrical charger 100, is mounted on the upper surface thereof.Next to the operating unit 114, there is mounted the electrical chargingunit 117 for electrically charging the charging battery as asub-battery.

FIG. 4 schematically shows the upper and lateral sides of the electricalcharger 100. As shown in FIG. 4, two markers, namely a main marker 118and a sub-marker 119, are provided to the electrical charger 100. Themain marker 118 is mounted at the uppermost location of the electricalcharger 100 so that it can be seen readily from the neighboring area.The height h of the uppermost point of the electrical charger 100 andthe distance a from the distal end of the electrical charger 100 to themain marker 118 are recorded in the robot apparatus 1 for use incalculating the distance between the robot apparatus. 1 and theelectrical charger 100. The color of the main marker 118 is not aroutine color such that the site of the main marker 118 can berecognized by the robot apparatus 1 detecting the color.

The sub-marker 119 is provided at the rear end of the electrical charger100. The color of the sub-marker 119 is also not a routine color suchthat the site of the sub-marker 119 can be recognized by the robotapparatus 1 detecting the color. The distance from the distal end of theelectrical charger 100 to the sub-marker 119 is recorded on the robotapparatus 1 for use in calculating the relative distance and therelative angle between the station coordinate system {s} and the robotcoordinate system {b} and in detecting possible mistaken markerrecognition.

The robot apparatus 1 is now explained. Referring to FIG. 5, leg units3A to 3D are connected to the front and rear sides and to the left andright sides of a body trunk unit 2. A head unit 4 and a tail unit 5 areconnected to the front and rear sides of the body trunk unit 2,respectively.

Referring to FIG. 6, the body trunk unit 2 is provided with a controlunit 16 and a battery 17. The control unit 16 is formed by a CPU(central processing unit) 10, a DRAM (dynamic random access memory) 11,a flash ROM (read-only memory) 12, a PC (personal computer) cardinterfacing circuit 13 and a signal processing circuit 14,interconnected over an internal bus 15. The battery 17 is a power supplyunit for the robot apparatus 1 that can be charged/dischargedrepeatedly. In the body trunk unit 2, there are stored an angularvelocity sensor 18 and an acceleration sensor 19 for sensing theorientation or the motion acceleration of the robot apparatus 1.

In preset locations of the head unit 4, there are mounted a CCD(charged-coupled device) camera 20 for imaging an external state, atouch sensor 21 for detecting the pressure applied by physical actions,such as [stroking] or [patting] from the user, a distance sensor 22 formeasuring the distance to a forwardly situated object, a microphone 23for collecting the external sound, a loudspeaker 24 for outputting thesound, such as pet's singing, and an LED (light emitting diode), notshown, equivalent to an ‘eye’ of the robot apparatus 1.

The joint portions of the leg units 3A to 3D, connecting portionsbetween the leg units 3A to 3D and the body trunk unit 2 and theconnecting portions between the head unit 4 and the body trunk unit 2,are provided with a number of actuators 25 ₁ to 25 _(n) and a number ofactuators 26 ₁ to 26 _(n) corresponding to the number of the requireddegrees of freedom. Each of the actuators 25 ₁ to 25 _(n) is providedwith for example a servo motor. Thus, the robot apparatus 1 may take atargeted orientation or perform targeted movements by the leg units,head unit 4 and the tail unit 5 being controlled by the driving of theservo motor.

The aforementioned various sensors, such as angular velocity sensor 18,acceleration sensor 19, touch sensor 21, distance sensor 22, microphone23, loudspeaker 24 or the potentiometers, and the actuators, areconnected via associated hubs 27 ₁ to 27 _(n) to the signal processingcircuit 14 of the control unit 16. The CCD camera 20 and the battery 17are directly connected to the signal processing circuit 14.

The signal processing circuit 14 sequentially takes in sensor data,image data or audio data, supplied from the above-mentioned respectivesensors, to store these data sequentially in preset locations in theDRAM 11 over the internal bus 15.

The signal processing circuit 14 sequentially takes in the residualbattery data, specifying the residual power to be supplied from thebattery 17, to store the data in preset locations in the DRAM 11.

The sensor data, image data, audio data and the residual battery data,thus stored in the DRAM 11, are used when the CPU 10 performsoperational control of the robot apparatus 1.

In an initial state when the power supply of the robot apparatus 1 isturned on, the CPU 10 reads out the control program stored in the flashROM 12 for storage in the DRAM 11. Alternatively, the CPU 10 reads outthe control program stored in a semiconductor memory device loaded in aPC card slot of the body trunk unit 2, not shown, such as in a so-calledmemory card 28, through a PC card interfacing circuit 13, for storage inthe DRAM 11.

The CPU 10 checks its own state, surrounding state, commands or actionsfrom the user, based on the sensor data, image data, audio data and theresidual battery data, thus stored in the DRAM 11 from the signalprocessing circuit 14, as described above.

Additionally, the CPU 10 determines the behavior which is based on theresults checked and on the control program stored in the DRAM 11. TheCPU 10 causes the necessary one(s) of the actuators 25 ₁ to 25 _(n),based on the determined results, to cause the head unit 4 to be swung inthe up-and-down direction or in the left-and-right direction, to causethe tail of the tail unit 5 to be moved, or to cause the leg units 3A to3D to walk.

The CPU 10 also generates speech data, if necessary, to send the sogenerated speech to the loudspeaker 24 through the signal processingcircuit 14. The CPU 10 also generates signals instructing on/off of theLED to turn the LED on or off. That is, the robot apparatus 1 behavesautonomously, responsive to the own or surrounding states, or commandsor actions from the user.

Since the robot apparatus 1 is able to behave autonomously, themovements of the robot apparatus 1 are obstructed if an electricalconducting line is connected to the robot apparatus 1 for supplying thepower thereto. Thus, the robot apparatus 1 is built-in the battery 17,as shown in FIG. 6, and is actuated by the power supplied from thisbattery 17. It is noted that the battery 17 is electrically charged bythe above-described electrical charger 100.

The robot apparatus 1 also has the automatic charging function, which isthe function the robot apparatus 1 has for autonomously finding theelectrical charger 100 when the residual battery capacity is about to bedepleted, and for performing the electrically charging operation foritself.

For executing this automatic charging function, a program concerningautomatic charging is stored in a storage device, such as the memorycard 28. The CPU 10 executes the automatic charging in accordance withthe operational sequence provided by this program. The processing forautomatic charging in the present embodiment of the robot apparatus 1 isnow explained specifically.

The processing for automatic charging is made up by a main marker searchprocessing for finding the main marker 118 provided at the uppermostpoint of the electrical charger 100, a sub-marker search processing forfinding the sub-marker 119 provided at the rear end of the electricalcharger 100, a distance calculating processing for calculating thedistance between the two markers 118, 119 and the robot apparatus 1,relative position calculating means for calculating the relativeposition/angle between the charger 100 and the robot apparatus 1, and apath determining processing for determining the path to an entrancepoint of the electrical charger 100.

In the main marker search processing, the robot apparatus 1 actuates itsfour limbs or neck to take in surrounding images to retrieve whether ornot there is contained the color information of the main marker 118 inthe thus taken-in image information, to recognize the presence of themain marker 118. The robot apparatus 1, which has recognized thepresence of the main marker 118, actuates the neck or the four limbs ofthe robot apparatus 1 to place the main marker 118 at the center of theimage.

The distance calculating processing is the processing of calculating thedistance between the markers 118, 119 and the robot apparatus 1. In thedistance calculating processing, the robot apparatus 1 sets fourcoordinate systems, namely a station coordinate system {s}, centeredabout the electrical charger 100, a robot coordinate system {b},centered about the trunk of the robot apparatus 1, a camera coordinatesystem {c}, centered about the CCD camera 20 provided to the head unitof the robot apparatus 1, and an imaging coordinate system {i}, havingthe center of the image photographed by the CCD camera 20 as a point oforigin. The height BCz from the point of origin B of the robotcoordinate system {b} to the camera coordinate system {c}, and acoordinate transformation matrix A_(CB) from the camera coordinatesystem {c} to the robot coordinate system {b}, are calculated andsubstituted into the following equation (1), to calculate the distancebetween the robot apparatus and the markers:

$\begin{matrix}{\alpha = \frac{{SP}_{s1x} - {BC}_{z}}{{A_{CB31}\frac{1}{f}h_{1}} + A_{CB32} + {A_{CB33}\frac{1}{f}v_{1}}}} & (1)\end{matrix}$where SPs1x is the distance from the point of origin S of the stationcoordinate system {s} to a center Ps1 of the main marker along they-axis, this value being pre-recorded in the robot apparatus 1 as thedistance a shown in FIG. 4.

This equation is calculated in accordance with the following algorithm:First, the above-described coordinate system is explained specificallyby referring to the drawings. FIG. 7 shows the four coordinate systemswith the electrical charger 100 and the robot apparatus 1 placed on aplanar floor. The station coordinate system {s} has a point of theelectrical charger 100 as the point of origin S. The x and y coordinatesare set on the floor where the electrical charger 100 is arranged, withthe z coordinate being the height from the floor. The robot coordinatesystem {b} has a point of intersection of a plumb line drawn from thecenter of gravity of the robot as the point of origin B. The x and ycoordinates are set on the floor, with the z coordinate being the heightfrom the floor. Thus, the xy plane of the station coordinate system {s}is the same as the xy plane of the of the robot coordinate system {b},such that, even if the coordinate transformation of the stationcoordinate system {s} and that of the robot coordinate system {b} aremade, the value of the z coordinate is not changed.

The camera coordinate system {c} has the CCD camera 20, mounted on thehead unit of the robot apparatus, as the point of origin C, with theviewing axis of the CCD camera 20 being the y-axis. The imagingcoordinate system {i} is two-dimensional coordinates set on an imagephotographed by the CCD camera 20, and has the center of thephotographed image as the point of origin I.

The algorithm for calculating the above equation (1) is explained. Inthis algorithm, the distance to a marker is set to α, and thedirectional vector of the marker in the imaging coordinate system {i} ismultiplied by this distance α to presume the position vector of themarker. An equation indicating the z-component (height of the marker) atthis time and an equation of the height h of the marker pre-stored inthe robot apparatus are formulated. The equation (1) for calculating thedistance α between the marker and the robot apparatus is found by termtransposition of the equation indicating the z-component.

The algorithm for finding the distance to the main marker 118 is nowexplained specifically. It should be noted that the algorithm forcalculating the distance of the main marker 118 is identified with thealgorithm for calculating the distance to the sub-marker 119, so thatthe algorithm for calculating the distance to the sub-marker 119 may bederived by changing the symbol Ps1 indicating the center position of themain marker 118 to the symbol Ps2 indicating the center position of thesub-marker 119. Thus, the explanation of the algorithm for calculatingthe distance of the sub-marker 119 is omitted.

First, if the coordinate of the center Ps1 of the main marker 118 in thecamera coordinate system {c} is (xc1, yc1, zc1), the directional vectorCPs1 from the point of origin C of the camera coordinate system to thecenter Ps1 of the main marker 118 is expressed by the following equation(2):

-   -   component of the [C] system of the directional vector of CP_(s1)

$\begin{matrix}{{CP}_{s1} = {\begin{bmatrix}\frac{x_{c1}}{y_{c1}} & 1 & \frac{z_{c1}}{y_{c1}}\end{bmatrix}^{T}.}} & (2)\end{matrix}$

If the x and z coordinates of Ps1 of the main marker 118 in the imagingcoordinate system {i} are h1, v1, respectively, the xz component of thedirectional vector CPs1 is transformed as indicated by the followingequation (3):

$\begin{matrix}\begin{matrix}{\frac{x_{c1}}{y_{c1}} = {\frac{1}{f}h_{1}}} \\{\frac{z_{c1}}{y_{c1}} = {\frac{1}{f}{v_{1}.}}}\end{matrix} & (3)\end{matrix}$

If, with the distance α from the point of origin C of the cameracoordinate system {c} to the center Ps1 of the main marker 118, thevector CPs1 proceeding from the point of origin C of the cameracoordinate system {c} to the center Ps1 of the main marker 118 is to beexpressed, this vector may be expressed by the following equation (4):

$\begin{matrix}{{\overset{arrow}{CP}}_{s1} = {{\alpha\begin{bmatrix}\frac{x_{c1}}{y_{c1}} & 1 & \frac{z_{c1}}{y_{c1}}\end{bmatrix}}^{T} = {{\alpha\begin{bmatrix}{\frac{1}{f}h_{1}} & 1 & {\frac{1}{f}v_{1}}\end{bmatrix}}^{T}.}}} & (4)\end{matrix}$

The position vector CPs1, indicating the center Ps1 of the main marker118 in the camera coordinate system {c}, is transformed into theposition vector BPs1, indicating the center Ps1 of the main marker 118in the robot coordinate system {b}, by the following equation (5):

$\begin{matrix}{{\overset{arrow}{SP}}_{s1} = {\begin{bmatrix}{SP}_{s1x} & {SP}_{s1y} & {SP}_{s1z}\end{bmatrix} = {A_{BS}{\overset{arrow}{BP}}_{s1}}}} & (5)\end{matrix}$where the vector BC is the position vector indicating the point oforigin C of the camera coordinate system {c} in the robot coordinatesystem {b} and A_(CB) is a coordinate transformation matrix from thecamera coordinate system {c} into the robot coordinate system {b}, ascalculated from the angle of the joints of the robot apparatus 1.

The equation (5), expressed by a matrix, is as shown in the followingequation (6):

$\begin{matrix}{\begin{bmatrix}{SP}_{s1x} \\{SP}_{s1y} \\{SP}_{s1z}\end{bmatrix} = {\begin{bmatrix}A_{BS11} & A_{BS12} & 0 \\A_{BS21} & A_{BS22} & 0 \\0 & 0 & 1\end{bmatrix}{( {\begin{bmatrix}{BC}_{x} \\{BC}_{y} \\{BC}_{z}\end{bmatrix} + {{\alpha\begin{bmatrix}A_{CB11} & A_{CB12} & A_{CB13} \\A_{CB21} & A_{CB22} & A_{CB23} \\A_{CB31} & A_{CB32} & A_{CB33}\end{bmatrix}}\begin{bmatrix}\begin{matrix}{\frac{1}{f}h_{1}} \\1\end{matrix} \\{\frac{1}{f}v_{1}}\end{bmatrix}}} ).}}} & (6)\end{matrix}$

On solving the equation (6), the z-component of the vector SPs1 is asshown by the following equation (7):

$\begin{matrix}{{SP}_{s1x} = {{BC}_{x} + {\alpha{\{ {{A_{CB31}\frac{1}{f}h_{1}} + A_{CB32} + {A_{CB33}\frac{1}{f}v_{1}}} \}.}}}} & (7)\end{matrix}$

By solving the equation (7) for α, the equation (1) may be calculated.As mentioned previously, the respective terms of the equation (1) are ofpre-known values, or of values calculated from the joint angles of therobot apparatus 1 or from the image information of the CCD camera 20.The distance α between the robot apparatus and the marker may becalculated by substituting these values into the equation (1).

The processing for calculating the sub-marker is executed using thedistance information calculated by the distance calculating processing.As mentioned previously, the main marker 118 is provided at theuppermost point of the electrical charger 100 and hence may readily befound by the robot apparatus 1. However, the sub-marker 119 is providedto the rear end of the robot apparatus and hence is hidden in the shadeof the electrical charger 100 so that the sub-marker may not be withinthe viewing field of the CCD camera 20 of the robot apparatus 1. Thus,in the sub-marker detection processing, the robot apparatus 1 is made towalk round the main marker 118 to search for the sub-marker 119.

In searching for the sub-marker 119, the robot apparatus 1 is caused tomake the round of the main marker 118 to delineate a regular octagonalshape therearound in the counterclockwise direction to shoot an image ateach apex of the regular octagon to search whether or not the colorinformation proper to the sub-marker 119 is included in the imageinformation. The procedure of causing the robot apparatus 1 to walk todelineate the shape of a regular octagon is as follows: On finding themain marker 118, the robot apparatus 1 searches whether or not the colorinformation proper to the sub-marker 119 is contained in the imageinformation. If the color information proper to the sub-marker 119 iscontained in the image information, the processing for searching thesub-marker is terminated.

If the image of the sub-marker 119 is not included in the photographedimage of the main marker 118, movement to an apex point of the regularoctagon is initiated. The robot apparatus 1, which has initiated itsmovement towards the apex points of the regular octagon, performs aturning movement so that the body of the robot apparatus 1 is orientedtowards the main marker 118. The distance between the main marker 118and the robot apparatus 1 is calculated by distance calculatingprocessing and the robot apparatus 1 is moved in the back-and-forthdirection until a preset distance is reached. Moreover, the robotapparatus 1 turns its body 67.5° in the clockwise direction and performsan advancing movement a distance equal to the length of a side of theoctagon for moving to the apex point of the octagon.

On getting to the apex point of the octagon, the robot apparatus 1 movesits four limbs and neck to photograph surrounding images to searchwhether or not the color information proper to the main marker 118 iscontained in the image information. If the presence of the main marker118 is recognized it is checked whether or not the color informationproper to the sub-marker 119 is included in the image containing themain marker 118. If there is no color information proper to thesub-marker 119 in the image containing the main marker 118, the robotapparatus 1 moves to the next apex point to search a location where themain marker 118 and the sub-marker 119 are included in one and the sameimage. The robot apparatus 1 reiterates this sequence of operations.When the robot apparatus has moved to the location where the main marker118 and the sub-marker 119 are included in one and the same image, thesub-marker search processing is terminated.

On finding the sub-marker, the robot apparatus performs the processingof calculating the relative position. This processing of calculating therelative position is the processing of calculating the relativeposition/angle between the station coordinate system {s} and the robotcoordinate system {b}. FIG. 8 shows the relative position/angle betweenthe station coordinate system {s} and the robot coordinate system {b}.Referring to FIG. 8, the relative position is the position vector fromthe point of origin B of the robot coordinate system {b} to the stationcoordinate system {s}, while the relative angle is the angle of rotationθ of the coordinate systems, namely the station coordinate system {s}and the robot coordinate system {b}.

In the relative position calculating means, the robot apparatus 1 firstfinds the positions vectors BPs1 and BPs2 of the main marker and thesub-marker, using the distance calculating processing. The relativeposition/angle of the point of origin B of the robot coordinate system{b} and the point of origin S of the station coordinate system {s} isthen found by substituting the position vectors BPs1 and BPs2 into thefollowing equations (8) and (9):

$\begin{matrix}{\overset{arrow}{M} = {{\overset{arrow}{BP}}_{s1} - {\overset{arrow}{BP}}_{s2}}} & (8) \\{\overset{arrow}{M} = {\frac{{SP}_{s1x}}{\overset{arrow}{M}}( \overset{arrow}{M} )}} & \; \\{\theta = {{a\;\tan\; 2( {{M(2)},{M(1)}} )} - {\frac{\pi}{2}.}}} & (9)\end{matrix}$

The path determining processing is the processing of determining theproceeding path of the robot apparatus by which the robot apparatus mayefficiently get to an entrance point EP of the electrical charger 100.The entrance point EP is a position suited for the robot apparatus 1 toproceed to the electrical charger 100, and is set towards the rear endof the electrical charger 100. FIG. 9 shows the entrance point EP of theelectrical charger 100 and passage points of the robot apparatus 1. InFIG. 9, there is shown a rectangular area delimited by dots. This areais determined depending on the width of the robot apparatus 1, referredto below as a contact area. If the center of gravity of the robotapparatus 1 is intruded into this area, the robot apparatus 1 iscontacted with the electrical charger 100.

More specifically, supposing that the robot apparatus 1 is located at aposition Q in FIG. 9, and the robot apparatus proceeds directly from thepoint Q to the entrance point EP, the proceeding path of the robotapparatus 1 is a straight line interconnecting the point Q and theentrance point. If the robot apparatus 1 is moved along this straightline, the robot apparatus 1 is intruded into the contact area, thusbeing contacted with the electrical charger 100.

Thus, in the proceeding processing, the area around the electricalcharger 100 is divided into five areas of from I to V and paths ofdetour are set depending on the position where the robot apparatus 1 islocated. If the robot apparatus 1 is located in the area I, the robotapparatus 1 tries to pass through the passage point A to proceed towardsthe entrance point EP. If the robot apparatus 1 is located in the areaII, the robot apparatus 1 passes through the passage point D to proceedtowards the entrance point EP.

If the robot apparatus 1 is located in the area III, and tries toproceed to the passage point A, the robot apparatus is intruded into thecontact area and is contacted with the electrical charger 100. Thus, ifthe robot apparatus 1 is located in the area III, the robot apparatuspasses through the passage point B and subsequently proceeds to thepassage point A and then to the entrance point EP. If the robotapparatus 1 is located in the area IV, and tries to proceed to thepassage point D, the robot apparatus is contacted with the electricalcharger 100. Thus, if the robot apparatus 1 is located in the area IV,it proceeds towards the entrance point EP through the passage points Cand D. Thus, the robot apparatus 1 is moved along the selectedproceeding path until it reaches the entrance point EP.

The operation of the robot apparatus in the autonomous electricalcharging is explained with reference to FIG. 10, which schematicallyshows the operation of the robot apparatus 1. When the residual batterycapacity is about to be depleted, the robot apparatus 1 begins itsautomatic electrical charging processing. On starting its automaticelectrical charging processing, the robot apparatus 1 first commencesthe main marker search processing to take-in images of various spatialareas as it moves its neck or legs. On finding the main marker 118 in astep S1, the robot apparatus 1 searches whether or not the colorinformation proper to the sub-marker 119 is contained in the imagecontaining the main marker 118. If the color information proper to thesub-marker 119 is not contained in the photographed image (step S2), thesub-marker search processing is initiated. Thus, the robot apparatus 1makes the round of the apex points of the regular octagon, centeredabout the main marker 118, until the robot apparatus gets to a positionfrom which the robot apparatus may view the sub-marker 119 (step S3).

When both the main marker 118 and the sub-marker 119 are viewable (stepS4), the robot apparatus 1 performs the processing of calculating therelative positions to calculate the relative position/angular betweenthe robot apparatus and the electrical charger 100, and proceeds towardsthe entrance point EP of the electrical charger 100 by the proceedingprocessing (step S5). On getting to the entrance point EP, the robotapparatus 1 proceeds towards the robot apparatus support 111 (step S6).When the robot apparatus is snugly received in the robot apparatussupport 111, it bends its four limbs 3A to 3D to have its batteryconnecting portion contacted with the connecting portion 115 of theelectrical charger 100 for electrical charging (step S7) to terminatethe automatic charging processing.

The above-described sequence of operations is further explained, usingthe flowchart of FIG. 11. On startup of the automatic chargingprocessing, the robot apparatus 1 first commences the main marker searchprocessing and has the surrounding images photographed as it moves itsneck or legs to make a search whether or not the color informationproper to the main marker 118 is included in the photographed images(step S11). If the information proper to the main marker 118 is found(step S12), the the robot apparatus moves its neck or the legs to adjustthe position of the CCD camera 20 so that the entire main marker 118will be demonstrated in the image.

The reason for having the main marker 118 demonstrated in its entiretyin the image is that it becomes thereby possible to find the centroid ofthe main marker 118 in the taken-in image it is therefore unnecessaryfor the main marker 118 to be arranged in the center of the image in thestrict sense of the term. It is however desirable, for diminishing thedistortion of the lens of the CCD camera 20, to adjust the position ofthe CCD camera 20 so that the main marker 118 is arranged at a locationas close to the center of the image as possible.

When the position of the CCD camera 20 has been adjusted so that themain marker 118 in its entirety is demonstrated in the image, the robotapparatus 1 executes the distance calculating processing for calculatingthe position vector of the main marker 118. Under the effect of neckcontrol of the robot apparatus 1, it takes some time until the positionvector of the main marker 118 is converged. Thus, the robot apparatus 1calculates the position vector of the main marker 118 from one frameoutput from the CCD camera 20 to another, to calculate the differentialnorm which is the difference between the norms of theconsecutive-position vectors. Using this differential norm, the robotapparatus 1 calculates the proper position vector. The condition used inadopting the position vector is that the position vector shall be theone calculated after lapse of 64 frames as from the visual recognitionof the main marker 118 and that the differential norm of the positionvector shall be convered within 1 mm for not less than five frames.

FIG. 12 upper depicts a graph showing the relationship between the normof the position vector and the number of frames after visual recognitionof the position vector, while FIG. 12 lower depicts a graph showing therelationship between the differential norm of the position vector andthe number of frames after visual recognition of the position vector. InFIG. 12 lower, the portion of the graph enclosed in a circle satisfiesthe condition that 64 frames have passed and that the norm difference ofthe position vector converges to 1 mm or less for not less than fiveframes. Thus, by adopting the position vector, calculated within theportion surrounded by the circle, as the correct position vector, it ispossible to avoid the detection error caused by the instabilities inneck control to obtain the position vector with high reliability (stepS13).

The robot apparatus 1 then actuates a neck to take in the surroundingimage to search whether or not the color information of the sub-marker119 is contained in the taken-in image information (step S14). If thesub-marker 119 has not been found in the position which has visuallyrecognized the main marker 118 (step S15: NO), the robot apparatus 1begins the sub-marker search processing to travel along the sides of theregular octagon centered about the main marker 118 (step S16) to executethe processing of step S11.

The robot apparatus 1, which has visually recognized the sub-marker 119(step S17), finds, from the position vector difference between the mainmarker 118 and the sub-marker 119, the distance between the main marker118 and the sub-marker 119, and compares the distance between the mainmarker 118 and the sub-marker 119 stored in the robot apparatus to thecalculated distance, to make a check whether or not the robot apparatus1 has made mis-recognition of the markers 118, 119 (step S18).

That is, the distance between the main marker 118 and the sub-marker 119is fixed, so that, if the robot apparatus 1 has recognized an objectother than the marker as being a marker, the distance between the mainmarker 118 and the sub-marker 119 differs from the actual distance.Thus, if the distance between the main marker 118 and the sub-marker 119is not comprised within a preset range (step S19: NO), the robotapparatus 1 recognizes that it has mis-recognized the marker andproceeds to step S11 to search the marker again. If the distance betweenthe main marker 118 and the sub-marker 119 is comprised within a presetrange, the robot apparatus 1 assumes that it is recognizing the correctmarker (step S19: YES) and proceeds to the processing in a step S20.

In the step S20, the robot apparatus 1 executes the processing ofcalculating the relative positions to find the relative position/anglebetween the robot coordinate system {b} and the station coordinatesystem {s} to shift to the processing in a step S21. Meanwhile, thecoordinate transformation from the robot coordinate system {b} to thestation coordinate system {s} may be facilitated by finding the relativeposition/angle between the robot coordinate system {b} and the stationcoordinate system {s}.

In a step S21, the coordinate of the point of origin B of the robotcoordinate system {b} in the station coordinate system {s} is found andthe value of this point of origin B is used to execute the proceedingprocessing. The proceeding processing is the processing of selecting theproceeding path which will not cause collision of the robot apparatuswith the electrical charger 100. The robot apparatus 1 is moved along aproceeding path as determined by the proceeding processing (step S22) toreach the entrance point EP (step S23: YES). The robot apparatus 1,which has reached the entrance point EP, makes a turning movement untilits body is oriented to a mid point between the main marker 118 and thesub-marker 119, to make fine adjustment of the position of the robotapparatus 1 (step S24).

If, in the step S23, the robot apparatus 1 has not reached the entrancepoint EP (step S23: NO), the processing transfers to the step S11 tore-initiate the search for the main marker 118.

If the position of the robot apparatus 1 has been determined, the robotapparatus 1 advances towards the electrical charger 100. At this time,the robot apparatus 1 calculates the distance between the main marker118 and the robot apparatus 1, using the distance calculatingprocessing. When the distance between the main marker 118 and the robotapparatus 1 is a preset distance, the movement of the robot apparatus 1ceases (step S25).

The robot apparatus 1 then lowers its waist to contact the connectingportion of the battery 17 of the robot apparatus 1 with the contactportion of the electrical charger 100 (step S26). It is noted that therobot apparatus support 111 of the electrical charger 100 isbowl-shaped, such that, if the orientation or the position of the robotapparatus 1 is slightly offset from the normal orientation or position,the belly portion of the robot apparatus 1 may be correctly loadedaccurately in the robot apparatus support 111.

By the above-described sequence of operations, the robot apparatus 1recognizes that the residual battery capacity has become depleted, sothat the robot apparatus 1 may proceed autonomously to the electricalcharger 100 where it may be electrically charged. Thus, the robotapparatus 1 may keep on to be in operation, as long as the batterycapacity allows, without the user having to offer his/her help.

If such processing is added which change the walking pattern of therobot apparatus until the robot apparatus gets to the electricalcharger, responsive to the residual battery capacity, the robotapparatus may be caused to act as if it is rushing, thus improving itsentertainment properties.

If the residual battery capacity is smaller, control may be made so thatthe robot apparatus will not be separated from the electrical charger.If conversely the robot apparatus is in the neighborhood of theelectrical charger, control may be managed such that the robot apparatuswill frequently perform brisk movements that may lead to acceleratedbattery consumption, even if the residual battery capacity is scanty.

The above-described sequence of operations is executed by the softwarewhich will be explained below. The software structure of the controlprogram in the robot apparatus 1 is shown in FIG. 13, in which a devicedriver layer 30 is the lowermost layer of the control program, and ismade up by a device driver set 31 comprised of plural device drivers.Each device driver is an object allowed to have direct access to thehardware used in an ordinary computer, such as one used in a CCD cameraor a timer, and performs processing responsive to interrupt from theassociated hardware.

A robotic server object 32 is a lowermost layer of the device driverlayer 30, and is made up by a device driver manager 35, comprised of asoftware set providing an interface for accessing the hardware, such asvarious sensors or actuators, and a designed robot 36, made up by asoftware set, supervising the mechanism of the robot apparatus 1.

A manager object 37 is made up by an object manager 38 and a servicemanager 39. The object manager 38 is a software set supervising thebooting or end of operation of the software included in the roboticserver object 32, middleware layer 40 and an application layer 41, whilethe service manager 39 is a software set supervising the interconnectionamong the respective objects based on the connection information amongthe respective objects stated in a connection file stored in the memorycard 28.

The middleware layer 40 is an upper layer of the robotic server object32 and is made up by a software set providing the basic functions of therobot apparatus 1, such as picture or speech processing.

The application layer 41, on the other hand, is an upper layer of themiddleware layer 40 and is made up by software set determining thebehavior of the robot apparatus 1 based on the results of processing bythe respective software sets making up the middleware layer 40.

Referring to FIG. 14, the middleware layer 40 is made up by arecognition system 60 and a recognition system 69. The recognitionsystem 60 includes a signal processing module 50 for noise detection 50for noise detection, a signal processing module 51 for temperaturedetection, a signal processing module. 52 for lightness detection, asignal processing module 53 for a sound scale recognition, a signalprocessing module 54 for distance detection, a signal processing module55 for orientation detection, a signal processing module 56 for a touchsensor, a signal processing module 57 for motion detection, a signalprocessing module 58 for color recognition, and an input semanticsconverter module 59, while the recognition system 69 includes an outputsemantics converter module 68, a signal processing module 61 fororientation management, a signal processing module 62 for tracking, asignal processing module 63 for motion reproduction, a signal processingmodule 64 for walking, a signal processing module 65 for recovery fromfalldown, a signal processing module 66 for LED lighting, and a signalprocessing module 67 for sound reproduction.

The respective signal processing modules 50 to 58 of the recognitionsystem 60 take in relevant ones of the sensor data, picture data and thespeech data, read out from the DRAM 11 by the virtual robot 33 of therobotic server object 32 and perform preset processing on the sotaken-in data to send the processed result to the input semanticsconverter module 59. For example, the virtual robot 33 is designed as acomponent responsible for transmitting/receiving or converting signals,under a preset communication protocol.

Based on the processed results applied from these signal processingmodules, the input semantics converter module 59 recognizes its ownstate, surrounding state, command from the user or the action by theuser, such as [noisy], [hot], [bright], [a ball detected], [a falldowndetected], [stroked], [patted], [the sound scales of do, mi and so onheard], [a moving object detected], or [an obstacle detected], andoutputs the recognized results to the application layer 41.

Referring to FIG. 15, the application layer 41 is made up by fivemodules, namely a behavior model library 70, an behavior switchingmodule 71, a learning module 72, a feeling model 73 and an instinctmodel 74.

In the behavior model library 70, there are provided, as shown in FIG.16, independent behavior models 70 ₁ to 70 _(n) in association withseveral pre-selected conditional items, such as [case where residualbattery capacity is diminished], [recovery from the falldown state],[case where an obstacle is to be avoided], [case where a feeling is tobe expressed] or [case where a ball has been detected].

When the results of the recognition are given from the input semanticsconverter module 59 or when a preset time has elapsed from the time thelast result of recognition was given, the above behavior models 70 ₁ to70 _(n) decide on the next behaviors to be taken, as reference is had toparameter values of the associated emotions stored in the feeling model73 or to parameter values of the associated desires held in the instinctmodel 74 to output the determined results to the behavior switchingmodule 71.

In the present embodiment, the respective behavior models 70 ₁ to 70_(n) use an algorithm, termed finite probability automation, as atechnique for determining the next behaviors, as shown in FIG. 17. Thisalgorithm is such a one in which the next one of the other nodes NODE₀to NODE_(n) to which transfer is to be made from one of the nodes NODE₀to NODE_(n) is probabilistically determined based on the transitionprobability values P₁ to P_(n) as set for each of the arcs ARC₁ toARC_(n) interconnecting the respective nodes NODE₀ to NODE_(n).

Specifically, the respective behavior models 70 ₁ to 70 _(n) eachinclude a status transition table 80, forming its own behavior model,for each of the nodes NODE₀ to NODE_(n), each in association with thenodes NODE₀ to NODE_(n), as shown in FIG. 18.

In this status transition table 80, input events (results of therecognition), representing the conditions of transition in a given nodeare entered in the column of the [input event name] in the order of thefalling priority, and further conditions for the transition conditionsare entered in the relevant rows of columns of the [data name] and [datarange].

Thus, in the node NODE₁₀₀, represented in the status transition table 80of FIG. 18, given the results of the recognition of [ball detected], theball [size] being in a range [from 0 to 1000], which is afforded alongwith the results of the recognition, represents the condition fortransition to the Other node. In similar manner, given the results ofthe recognition of [obstacle detected], the [distance] to the obstacle,afforded along with the results of the recognition, being in a range[from 0 to 100], represents the condition for transition to the othernode.

Moreover, if, in this node NODE₁₀₀, there is no input of the results ofrecognition, but any of the values of the parameters [joy], [surprise]and [sadness], held by the feeling model 73, among the parameters of theemotions and desires, held by the feeling model 73 and the instinctmodel 74, periodically referenced by the behavior models 70 ₁ to 70_(n), is in a range from [50 to 100], transition may be made to theother node.

In the status transition table 80, the names of the nodes, to whichtransition may be made from the node NODE₀ to node NODE_(n), are enteredin the row [node of destination of transition] in the column [transitionprobability to the other node], while the transition probabilities tothe other node of the node NODE₀ to NODE_(n), to which transition may bemade when all the conditions entered in the columns of the [input eventname], [data name] and [data range] are met, are entered in the relevantcells of the column [transition probability to the other node]. Alsoentered in the row [output behavior] in the column [transitionprobability to the other node] are the behaviors to be output in makingtransition to the other of the nodes NODE₀ to NODE_(n).

Meanwhile, the sum of the probabilities of the respective rows in thecolumn [transition probability to the other node] is 100%.

Thus, in the node NODE₁₀₀, indicated in the status transition table 80of FIG. 18, if the results of the recognition are such that the [ball isdetected] and the [size] of the ball is in a range from [0 to 1000],transition may be made to the [node NODE₁₂₀] at a probability of [30%]and the behavior [ACTION 1] is taken at this time.

Each behavior model 70 ₁ to 70 _(n) is constructed that a number of thenodes NODE₀ to the node NODE_(n), stated in the status transition table80, are concatenated, such that, when the results of the recognition areafforded from the input semantics converter module 59, the next behavioris determined probabilistically by exploiting the status transitiontable of the corresponding nodes NODE₀ to NODE_(n), with the results ofthe decision being output to the behavior switching module 71.

The behavior switching module 71, shown in FIG. 15, selects the outputbehavior from the behaviors output from the behavior models 70 ₁ to 70_(n) of the behavior model library 70 so that the behavior selected isone output from the predetermined behavior model with the highest rankin the priority order. The behavior switching module 71 sends a commandfor executing the behavior, referred to below as the behavior command,to an output semantics converter module 68 of the middleware layer 40.Meanwhile, in the present embodiment, the behavior models shown in FIG.16 becomes higher in the descending direction in the drawing.

Based on the behavior completion information, afforded from the outputsemantics converter module 68 after the end of the behavior, thebehavior switching module 71 informs the learning module 72, feelingmodel 73 and the instinct model 74 of the end of the behavior.

The learning module 72 inputs the results of the recognition of theinstructions, received as the action from the user, such as [patting] or[stroking], among the results of the recognition afforded from the inputsemantics converter module 59.

Based on the results of the recognition and on the notice from thebehavior switching module 71, the learning module 72 changes thecorresponding transition probability of the corresponding one of thebehavior models 70 ₁ to 70 _(n) in the behavior model library 70 forlowering and raising the probability of occurrence of the behavior incase of patting (scolding) and stroking (praising), respectively.

On the other hand, the feeling model 73 holds parameters indicating theintensity of each of six emotions of [joy], [sadness], [anger],[surprise], [disgust] and [fear]. The feeling model 73 periodicallyupdates the parameter values of these emotions based on the specifiedresults of the recognition afforded by the input semantics convertermodule 59, such as [patting] or [stroking], time elapsed and on noticesfrom the behavior switching module 71.

Specifically, the feeling model 73 calculates, based on the results ofthe recognition supplied from the input semantics converter module 59,the behavior of the robot apparatus 1 at this time and on the timeelapsed since the previous update operation, a parameter value E[t+1] ofa given emotion in the next period by the equation (10):E[t+1]=E[t]+k_(e)×ΔE[t]  (10)where ΔE(t) is the variation of the emotion as calculated by a presetequation for calculation, E[t] is the current parameter value of theemotion, and k_(e) is the coefficient representing the sensitivity ofthe emotion, and substitutes the parameter value E[t+1] for the currentparameter value of the emotion E[t] to update the parameter value of theemotion. The feeling model 73 also updates the parameter values of thetotality of the emotions in similar manner.

Meanwhile, to which extent the results of the recognition or the noticefrom the output semantics converter module 68 affect the amount of thevariation ΔE[t] of the parameter values of the respective emotions ispredetermined, such that the results of the recognition [being patted]seriously affect the amount of the variation ΔE[t] of the parametervalue of the emotion [anger], while the results of the recognition[being stroked] seriously affect the amount of the variation ΔE[t] ofthe parameter value of the emotion [joy].

It is noted that the notice from the output semantics converter module68 is what may be called the feedback information of the behavior(behavior end information), that is the information concerning theresults of the occurrence of the behavior, and that the feeling model 73changes its emotion by this information. For example, the behavior of[barking] lowers the feeling level of anger.

Meanwhile, the notice from the output semantics converter module 68 isalso input to the learning module 72 such that the learning module 72changes the corresponding transition probability of the behavior models70 ₁ to 70 _(n) based on such notice. Meanwhile, the feedback of theresults of the behavior may be made by the output of the behaviorswitching module 71 (behavior added by the feeling).

The instinct model 74 holds parameters, indicating the strength of fourindependent desires, for example, the [desire for exercise], [desire foraffection], [appetite] and [curiosity]. Based on the results of therecognition afforded by the input semantics converter module 59, timeelapsed and on the notice from the behavior switching module 71, theinstinct model 74 periodically updates the parameters of these desires.

Specifically, the instinct model 74 updates, for the [desire forexercise], [desire for affection] and [curiosity], based on the resultsof the recognition, time elapsed and on the notice from the outputsemantics converter module 68, the parameter value of the desire inquestion by calculating, at a preset period, a parameter value for thedesire in question I[k+1] for the next period, using the followingequation (11):I[k+1]=I[k]+k_(i)×ΔI[k]  (11)where ΔI[k] is the amount of the variation of the desire as calculatedby a preset equation for calculation,. I[k] is the current parametervalue of the desire in question and k_(i) is the coefficient expressingthe sensitivity of the desire in question, and by substituting theresults of the calculation for the current parameter value I[k] of thedesire in question, the parameter value of the desire in question isupdated. In similar manner, the instinct model 74 updates the parametervalues of the respective desires different than the [appetite].

Meanwhile, to which extent the results of the recognition and the noticefrom the output semantics converter module 68 affect the amount of thevariation ΔI[k] of the parameter values of the respective desires ispredetermined, such that the results of the recognition seriouslyaffects the amount of the variation ΔI[k] of the parameter value of the[fatigue].

In the present embodiment, the parameter values of the respectiveemotions and desires (instincts) are controlled to be varied in a rangefrom 0 to 100, while the values of the coefficients k_(e) and k_(i) areset from one emotion to another and from one desire to another.

On the other hand, the output semantics converter module 68 of themiddleware layer 40 gives abstract behavioral commands afforded by thebehavior switching module 71 of the application layer 41, such as[advance], [joy], [speak] or [tracking (track a ball)], to the signalprocessing modules 61 to 67 of the output system 69, as shown in FIG.14.

If a behavioral command is issued, the signal processing modules 61 to67 generates data such as servo command values, based on the behavioralcommand, which are to be supplied to the associated actuator 25 ₁ to 25_(n) for performing the behavior, to send these values or data to theassociated actuator 25 ₁ to 25 _(n), through the virtual robot 33 of therobotic server object 32 and the relevant signal processing circuit 14.

The signal processing modules 61 to 67 generate speech data of the soundoutput from the loudspeaker 24 and/or data pertinent to the operation ofthe LED operating as an [eye] to send these data sequentially to theloudspeaker 24 or the LED through the virtual robot 33 of the roboticserver object 32 and the relevant signal processing circuit 14.

In this manner, the robot apparatus 1 is able to perform autonomousbehavior, responsive to its own inner status, surrounding (external)status and commands or actions from the user, based on theaforementioned control program.

While the invention has been described in accordance with certainpresent embodiments thereof illustrated in the accompanying drawings anddescribed in the above description in detail, it should be understood bythose ordinarily skilled in the art that the invention is not limited tothe embodiments, but various modifications, alternative constructions orequivalents can be implemented without departing from the scope and thespirit of the present invention as set forth and defined in the appendedclaims.

1. An electrical charging system including a robot apparatus and anelectrical charging device, said robot apparatus having built-in a powersupply unit for driving and adapted for acting autonomously responsiveto an inner state, said electrical charging device electrically chargingsaid power supply unit, wherein said electrical charging device includesat least one marker provided at a predetermined position; said robotapparatus including imaging means for imaging a surrounding state andmovement means to move said robot apparatus; storage means having theinformation on the position of said marker pre-stored therein; positionvector calculating means for calculating the distance and the directionto said marker from the information on said marker position stored insaid storage means and from an image including the marker position,photographed by said imaging means; and operation controlling means forperforming control for driving said movement means, causing movement ofsaid robot apparatus towards said electrical charging device dependingon the distance and the direction to said marker calculated by saidposition vector calculating means; wherein said at least one markercomprises a main marker and a sub marker, the main marker is provided atan uppermost location of the electrical charging device and the submarker is provided at a rear end of the electrical charging device.